5.1 Quality Control and Quality Assurance Procedures

QA/QC procedures included, crosschecking at Surveyor level, 100% administrative checks, and 20% field checks by team leaders, 100% administrative checks by field editors, 1% field checks by the Field Managers, 3% checks by the QA/QC team, 100% checks by the data entry staff, and a number of special investigations by the QA/QC team in order to resolve apparent anomalies.

Mick McDonell (left) and Chau Touch San

The QA / QC Manager was employed from November 2000 till the end of field operations in April 2002. Over the project period he and his assistant, pictured above, visited villages after they had been surveyed in order to produce field assessments.

Some assessments later became redundant due to resurvey activity, as a consequence 421 of the 486 village field assessments made by the QA / QC team are recorded in the survey database. This represents 3.0% of the total surveyor output.

The survey was designed with the following three integrated procedural elements of Quality Control (QC) and Quality Assurance (QA):

Field Survey procedures carried out by the field survey staff, responsible to the Field Project Manager, through the Field Survey Manager.

Database procedures carried out by the Database staff, responsible to the Field Project Manager, through the Database Manager.

Quality Assurance procedures carried out by the QA team, responsible to the Field Project Manager, through the QA Manager.

The management team promoted the understanding by all national and international staff that the product would only ever be as good as the information collected, and that it was the responsibility of all staff to promote quality.

Process

The combined elements of Quality Assurance and Quality Control are illustrated below. The strength and flexibility of the process is the open passage of information providing prompt feedback that allows for timely remedial action.

Survey Quality Assurance and Quality Control Process

Coordination with Field Staff / Interaction with the Survey Database

Close coordination and co-operation between the Survey Field Staff, the QA / QC team and the Database staff ensured that the maximum benefit was achieved. Effective communications between these three elements meant that issues were identified and dealt with in a timely fashion, the result being a continual reduction in the number of errors occurring, and a high quality of information gathered.

Refresher Training

The most effective feedback to the surveyors was the periodic refresher training conducted by the Field Survey Managers, with contributions by QA and Database Units. The quality and the consistency of this training was a critical factor in the success of the survey.

Documentation

Record keeping and progressive reporting was carried out throughout the survey period. Reporting included but was not limited to:

• All managers: monthly activity reporting, accidents, incidents and investigations,



• Field staff: periodic monitoring reports, refresher training reports, progress reports,



• Database: progress reports, correction reports, rejected (queried) survey reports, and



• Quality Assurance: periodic activity, progress, status, distribution and a log of villages visited.

QA assessments conducted in the field were documented on the Field Quality Assurance Check list. The checklist was complied in part by the QA Assistant Manager, and completed by the QA Manager then dated and signed by both team members. These documents are on file in the survey database.

Quality Control and Quality Assurance operating procedures employed during the survey are detailed in
SOP 11, SOP 12, and SOP 16. UNMAS certification guidelines are included in the Impact Survey Certification Committee Terms of Reference Annex B. (Impact Survey Certification Guidelines and Document Requirements.)

Field Survey Staff Quality Control Procedures

Systematic checks were conducted within the field operations, at all operational levels.

Surveyors

The surveyors worked to ensure that accurate information was being gathered, surveyors conducted crosschecks on the work of their detachment partner throughout the village survey and documentation compiling process. This included on occasion, revisits to the village as part of the survey process to confirm information and or to correct omissions.

In addition surveyors were employed on an as required basis to assist the team leaders to conduct their 100% review of the reports prior to submission to the Field Survey Office.

Kampong Cham (Translated by Mom Chan Komoth) Sometimes villagers' sense of humour puts a new perspective on the questionnaire. We are interested in the history of conflict in an area, but in Cambodia conflict sometimes took place thirty years ago and people find it difficult to answer the questions. When we asked the chief of village, which parties bombed his village he clearly could not remember. Instead of saying that he did not know he responded by saying, "who knows who they were in the sky...!" (By: Norm Sinath, Team Leader T4 and Surveyors)

Team Leaders

Team Leaders were responsible for the continuous monitoring of the surveyors under their control, carrying out a 100% review of all reports submitted and field checks of at least 20% of completed survey questionnaires.

Team Leaders made corrections to the reports and periodically provided the field managers with a narrative monitors report detailing the results of checks conducted in the field, the nature of errors occurring, and remedial actions taken.

Field Editors

The original project design did not include Field Editors, instead the responsibility was placed on the team leaders to conduct 100% administrative checks and 20% field checks. It was identified early in the project that this was non-effective and placed a severe burden on the team leaders. The corrective action taken was to identify two surveyors who had displayed an aptitude for the work and place them in the Field Office to assist the Field Survey Manager to check for errors and omissions. The field editors collated and compiled Provincial, District, and Commune summaries, packaged and ensured the security of the reports.

Similarly each team employed surveyors as field editors to assist the team leaders at the team accommodation prior to submitting the report to the Field Office.

Field Survey Manager and Assistant Field Survey Manager

The Field Survey Manager and the Assistant Field Survey Manager were responsible for ensuring a high quality survey result. This entailed ensuring that all previously described aspects of field survey quality control were carried out.

The field managers monitored survey rates and survey results and advised on changes to survey procedures and approaches as necessary and appropriate, carrying out field checks of at least 1% of all completed survey questionnaires.

Feedback and Follow Up

The Field Survey Manager and the Assistant Field Survey Manager supported by the Quality Assurance and Database managers periodically provided refresher training to the surveyors in aspects of information gathering that had been identified by Quality Assurance, Database, or internal Quality Control as creating errors in the reports.

Database Procedures

Survey reports arriving at the Database faced a comprehensive examination of all data collected.

Initial checks were conducted to make sure that all the reports were correctly compiled and complete then checks were conducted for typographical errors, in all sections of the report.

Database Corrections

The database was designed with a logical framework that prevented incomplete or illogical entries. Simple administrative errors were corrected at the point of input; typical errors were missing information such as the age of a respondent, detachment number, or an undated report. At the time of reporting the database had records of 764 corrections made in 235 reports.

GIS Information

100% checks were made by the GIS Operator and the Database Manager on village and area sketches, when errors were identified the report was returned through the Quality Assurance Manager to the field for correction. This procedure proved to be invaluable for providing the field staff with feedback on the number and nature of errors occurring.

Database Feedback and Follow Up

The Database provided feedback to the surveyors and followed up on reports containing errors in three ways:

• Periodic notification to each detachment on the errors identified by the database staff in the reports submitted, this mainly related to administrative type errors.



• The processing of reports queried by the Database Manager back to the field for verification or correction. Not all reports processed in this way contained errors, but were put back to the field for clarification.



• Collective refresher training and presentations by database staff to the surveyors.

Quality Assurance Procedures

The Quality Assurance team, independent of other field operations, conducted field assessments on 3% of the village survey reports. A Quality Assurance checklist was developed in order to ensure a simple and standardised procedure. Four routines were developed to deal with the different circumstances encountered in the field:

• Preliminary routine, for all field checks, this included checks and transcription of all village information from the report onto the checklist. Checks were made by the assistant to determine the security situation, access, and other relevant information relating to the safe and efficient conduct of the field check.



• Alpha routine, conducted in villages where the report indicated no mine / UXO contamination.



• Bravo routine, in villages where the report indicated mine / UXO contamination but where circumstances did not allow a complete check, most commonly when the key respondent, the village leader or other representative was unavailable for the interview.



• Charlie routine, in villages where the report indicate mine / UXO contamination and where there were no restrictions on the conduct of the field check.

On every occasion the compliance with procedures and the performance of the survey detachment was assessed, the use of the checklist and the adoption of the routines outlined above allowed for accurate record keeping and analysis of surveyor performance.

Management Feedback and Follow Up

QA follow up procedures provided feedback to the management and the staff that conducted the survey and took three forms:

• Immediate feedback to the team leaders and detachments in the field at the time the check was conducted including a performance statement.



• Periodic progress reports to the Field Project Manager, Field Survey Manager, and Assistant Survey Manager, Database Manager, and Team Leaders.



• Periodic reporting to the collective survey staff, outlining QA progress, status, and feedback on staff performance.

Back to Section 2.1 Inception Mission
Next Section - 5.2 Application and Results
Back to Table of Contents